A production environment having a large number of servers and data centers distributed geographically would benefit from a capability of testing groups of components or systems in various environments. Types of testing can include system testing, integration testing, and end-to-end testing. Integration testing is a type of testing in which two or more sub-components are combined and tested as a group. Integration testing includes validating the interface, data contract and functionality between sub-components. Integration testing typically occurs after unit testing of the sub-components. End-to-end testing is performed by applying tests to a group of various systems, from start of a given process to finish and validation at both points, including intermediate validation at given steps.
The testing process may range from deployment of binaries locally against code built from a mainline system, to clusters for load or integration testing, and to production for larger system testing or data validation.
Much manual overhead is involved in maintaining large, production-like systems. The production-like systems are manually deployed, maintained, refreshed, and typically become stale. Engineering organizations may also deploy system tests by pointing to production binaries. This limits the ability to simulate test data or scenarios or point to mainline-built binaries.